45 research outputs found
Stochastic homogenization of nonconvex Hamilton-Jacobi equations: a counterexample
We provide an example of a Hamilton-Jacobi equation in which stochastic
homogenization does not occur. The Hamiltonian involved in this example
satisfies the standard assumptions of the literature, except that it is not
convex
A Tauberian theorem for nonexpansive operators and applications to zero-sum stochastic games
We prove a Tauberian theorem for nonexpansive operators, and apply it to the
model of zero-sum stochastic game. Under mild assumptions, we prove that the
value of the lambda-discounted game v_{lambda} converges uniformly when lambda
goes to 0 if and only if the value of the n-stage game v_n converges uniformly
when n goes to infinity. This generalizes the Tauberian theorem of Lehrer and
Sorin (1992) to the two-player zero-sum case. We also provide the first example
of a stochastic game with public signals on the state and perfect observation
of actions, with finite state space, signal sets and action sets, in which for
some initial state k_1 known by both players, (v_{lambda}(k_1)) and (v_n(k_1))
converge to distinct limits
The Complexity of POMDPs with Long-run Average Objectives
We study the problem of approximation of optimal values in
partially-observable Markov decision processes (POMDPs) with long-run average
objectives. POMDPs are a standard model for dynamic systems with probabilistic
and nondeterministic behavior in uncertain environments. In long-run average
objectives rewards are associated with every transition of the POMDP and the
payoff is the long-run average of the rewards along the executions of the
POMDP. We establish strategy complexity and computational complexity results.
Our main result shows that finite-memory strategies suffice for approximation
of optimal values, and the related decision problem is recursively enumerable
complete
General limit value in zero-sum stochastic games
Bewley and Kohlberg (1976) and Mertens and Neyman (1981) have proved,
respectively, the existence of the asymptotic value and the uniform value in
zero-sum stochastic games with finite state space and finite action sets. In
their work, the total payoff in a stochastic game is defined either as a Cesaro
mean or an Abel mean of the stage payoffs. This paper presents two findings:
first, we generalize the result of Bewley and Kohlberg to a more general class
of payoff evaluations and we prove with a counterexample that this result is
tight. We also investigate the particular case of absorbing games. Second, for
the uniform approach of Mertens and Neyman, we provide another counterexample
to demonstrate that there is no natural way to generalize the result of Mertens
and Neyman to a wider class of payoff evaluations
Constant payoff in zero-sum stochastic games
In a zero-sum stochastic game, at each stage, two adversary players take
decisions and receive a stage payoff determined by them and by a random
variable representing the state of nature. The total payoff is the discounted
sum of the stage payoffs. Assume that the players are very patient and use
optimal strategies. We then prove that, at any point in the game, players get
essentially the same expected payoff: the payoff is constant. This solves a
conjecture by Sorin, Venel and Vigeral (2010). The proof relies on the
semi-algebraic approach for discounted stochastic games introduced by Bewley
and Kohlberg (1976), on the theory of Markov chains with rare transitions,
initiated by Friedlin and Wentzell (1984), and on some variational inequalities
for value functions inspired by the recent work of Davini, Fathi, Iturriaga and
Zavidovique (2016